SignalGuru: Leveraging mobile phones for collaborative traffic signal schedule advisory

While traffic signals are necessary to safely control competing flows of traffic, they inevitably enforce a stop-and-go movement pattern that increases fuel consumption, reduces traffic flow and causes traffic jams. These side effects can be alleviated by providing drivers and their onboard computational devices (e.g., vehicle computer, smartphone) with information about the schedule of the traffic signals ahead. Based on when the signal ahead will turn green, drivers can then adjust speed so as to avoid coming to a complete halt. Such information is called Green Light Optimal Speed Advisory (GLOSA). Alternatively, the onboard computational device may suggest an efficient detour that will save the driver from stops and long waits at red lights ahead. This paper introduces and evaluates SignalGuru, a novel software service that relies solely on a collection of mobile phones to detect and predict the traffic signal schedule, enabling GLOSA and other novel applications. Our SignalGuru leverages windshield-mounted phones to opportunistically detect current traffic signals with their cameras, collaboratively communicate and learn traffic signal schedule patterns, and predict their future schedule. Results from two deployments of SignalGuru, using iPhones in cars in Cambridge (MA, USA) and Singapore, show that traffic signal schedules can be predicted accurately. On average, SignalGuru comes within 0.66s, for pre-timed traffic signals and within 2.45s, for traffic-adaptive traffic signals. Feeding SignalGuru's predicted traffic schedule to our GLOSA application, our vehicle fuel consumption measurements show savings of 20.3%, on average.National Science Foundation (U.S.). (Grant number CSR-EHS-0615175)Singapore-MIT Alliance for Research and Technology Center. Future Urban Mobilit

Personalizing Audio Content Played While On Hold During a Phone Call

Calls made to a business by a customer, e.g., to request support, are often put into a queue waiting for a human agent to be available. During the hold time, canned music or other audio is played back to the caller. Such audio is low quality owing to the limited capacity of the telephony channel, is not personalized, and repeated multiple times till an agent becomes available, providing an unsatisfactory calling experience. This disclosure describes the use of machine learning techniques to detect canned audio and replace it with high fidelity music or other content. With user permission, the replacement content can be personalized, e.g., based on a user’s music playlists/preferences, and context. Machine learning techniques can also be utilized to upscale music on hold experience provided by the business. With user permission, advertising content or helpful content about the business can be delivered during the hold time. The techniques can be integrated into a virtual assistant or device operating system to provide an improved calling experience

Using Code Review Repositories and Changelists to Train Large Language Models for Code Generation

While large language models (LLMs) can generate code, training of such models has not made use of data generated during the collaborative code review process that is a standard part of software development. This disclosure describes techniques that utilize historical code review data (including reviewer comments and corresponding code edits) available within organization internal code repositories to train LLMs to generate code. The historical code review data can be used for model tuning, to train an LLM via reinforcement learning from human feedback (RLHF), and/or via prompt engineering. The trained model can be utilized to generate code starting from code description provided using a prompt template. The prompt template can incorporate organization specific factors such as developer guidelines, developer or team style, etc. Code generated by the LLM can be iteratively refined via human review as well as from analytical tools that ensure style compliance, code coverage, test success rate, comment conventions, etc

AI-based Image Synthesis for Enriched Search and Shopping

Across numerous applications, notably in search and shopping for unique items, humans are constrained by what has already been built or designed. This disclosure describes techniques that leverage natural language-based, deep-learning image synthesis to deliver enhanced product search via services such as search engines or e-commerce websites. The synthetically generated products can be custom manufactured upon order. Unconstrained by real world objects, the techniques deliver to the search engine or e-commerce user synthetic objects based on text descriptions provided by the user

User Customizable Content Summarization Using Deep Learning

Users expend substantial time and mental effort in trying to distill the most important facts and/or insight from lengthy text content. While deep learning models can summarize long documents, such models typically operate on the backend. Model output is not customizable by end users via simple user interfaces. This disclosure describes mechanisms that provide users with the ability to obtain customized automatic summaries of documents. Users can specify the desired properties of the summaries based on their preferences and constraints such as available time, desired length or style for the summary, output language, complexity, etc. via convenient UI elements. Users can also choose to obtain automatically generated images or other media that summarize the document content. Customization of output can make the automatically generated summary more relevant and useful and save the user time and effort in reading lengthy text

Improving Remote Customer Interaction Experiences Using Machine Learning

A common problem in contact centers is high employee turnover. Artificial intelligence (AI) techniques that have been introduced to smoothen interaction and improve the customer’s experience can have opposite effects, e.g., by requiring the customer to navigate complex menu options. This disclosure describes AI-based techniques applied to agent training and customer calls. The techniques can reduce turnover at contact centers and improve the experience of end users who interact with customer service agents. Per the techniques, suitable AI techniques are implemented to train human customer agents, and human feedback is in turn used to train AI techniques. Human-AI augmentation can be used to mirror the communication styles of customers to improve the interaction experience. The techniques can also be used to improve safety, e.g., by automatically detecting scam calls and alerting users. The techniques enable the creation of scalable, standalone, artificial or human-AI augmented customer service agents

Collaborative and Adaptive Mobile Device-resident Service Architectures

Mobile devices, such as smartphones and personal media players, have recently significantly increased in popularity thanks to the rich set of mobile cloud services that they allow users to access. Networked vehicular computing devices are also expected to be commonplace in the near future, as they will enable a wide range of driver assistance services. The ubiquitous penetration of mobile services, however, has been thwarted by their poor user experience; access to mobile cloud services typically occurs over slow and costly long-range cellular communications. This thesis focuses on improving the user experience of mobile services by reducing the need for costly long-range cellular communications. To achieve this, the thesis proposes to host more service functionality on mobile devices themselves. In this way, mobile devices are often able to serve requests either locally or by contacting neighbor devices over short-range communications. Two novel mobile service architectures are proposed for the two different types of mobile services: traditional non-geo-locality services and emerging geo-locality services. A service is termed to have the geo-locality property when its data are both generated (sensed or input) and consumed locally, i.e., within a specific geographic region. In other words, for services that have the geo-locality property, only mobile devices within a specific geographic region R can generate the necessary service data and only devices within the very same region R are interested in consuming it. For non-geo-locality services, the data is generated either by cloud servers or by users regardless of their location. Data generation and/or consumption are also typically a function of the users' personal interests and not of their geographic location. For traditional non-geo-locality services, this thesis proposes the Pocket Cloudlets architecture. The Pocket Cloudlets architecture is a mobile device-resident caching scheme that serves cloud service requests locally on the device, when possible, significantly reducing the need for slow and costly long-range communications. The Pocket Cloudlets architecture leverages both personal user and collaboratively-generated community access patterns to selectively replicate parts of the cloud service locally on the mobile device. Pocket Cloudlets are also adaptively updated by detecting emerging popular service data items and prefetching them on the mobile device. Our analysis shows that the proposed Pocket Cloudlets architecture can effectively augment several traditional cloud services, like mobile web search. PocketSearch, our prototyped mobile search pocket cloudlet, reduces the average service access time by a factor of 2.7× and the required communication bandwidth by 66%. For emerging geo-locality services, the thesis presents the Region-Resident Services (RegReS) middleware. RegReS allows a rich set of emerging geo-locality services to be fully supported on confederations of mobile devices. Mobile devices collaborate to provide a geo-locality service within a specified region and over a specified service lifetime by utilizing only short-range ad-hoc communications. In this way, RegReS completely eliminates the need for long-range cellular communications. Although mobile devices are becoming increasingly powerful, their resources are constrained and should be used judiciously. RegReS enables the efficient provision of geo-locality services by allowing services to specify their target service carrier density. Only as many service carriers as specified are subsequently maintained by RegReS. As opposed to previously proposed static schemes, RegReS employs a fully distributed, collaborative and adaptive estimation scheme to track the existing service carrier density and make decisions about the spawning of new carriers, when necessary. Thanks to collaboration and adaptation, RegReS can maintain the desired density with only 16% mean absolute error across a wide range of configurations. To demonstrate the potential of collaborative mobile device-based computing platforms that are enabled by middleware like RegReS, the thesis presents a rich set of novel services that such platforms can enable. More specifically, the thesis focuses on the type of services that are typically most challenging and resource-intensive (e.g., CPU), camera-based services. We introduce five such services and prototype SignalGuru, a camera-based traffic signal schedule advisory service. SignalGuru leverages opportunistic sensing and collaboration across windshield-mounted smartphones and their cameras to provide drivers with information about the schedule of traffic signals ahead. Results from two deployments of SignalGuru, using iPhones in cars in Cambridge (MA, USA) and Singapore, show that traffic signal schedule can be predicted with very good accuracy. On average, SignalGuru comes within 0.66s, for pre-timed traffic signals and within 2.45s, for traffic-adaptive traffic signals. Feeding SignalGuru's predicted traffic schedule to our Green Light Optimal Speed Advisory (GLOSA) application, our vehicle fuel consumption measurements show savings of 20.3%, on average. SignalGuru information can also be fed into several other envisioned applications to further improve fuel efficiency, vehicle flow, travel time and road safety. The example of SignalGuru illustrates that with collaboration and adaptation, mobile device-based computing platforms can support a rich set of challenging services without the help of cloud servers and the associated long-range communications. Overall, this thesis advocates and demonstrates that, with collaboration and adaptation, mobile devices can effectively support a rich set of services and thus reduce the need for slow and costly long-range cellular communications to cloud servers. Several traditional cloud services that operate on very large amounts of data can be selectively and adaptively hosted on mobile devices. Furthermore, novel mobile services that may seem prohibitively resource-intensive and challenging can be enabled and hosted on confederations of collaborating mobile devices. In this way, the mobile user experience can be greatly improved and a significant amount of the increasingly scarce long-range communication bandwidth can be saved

Leveraging Smartphone Cameras for Collaborative Road Advisories

Ubiquitous smartphones are increasingly becoming the dominant platform for collaborative sensing. Smartphones, with their ever richer set of sensors, are being used to enable collaborative driver-assistance services like traffic advisory and road condition monitoring. To enable such services, the smartphones' GPS, accelerometer, and gyro sensors have been widely used. On the contrary, smartphone cameras, despite being very powerful sensors, have largely been neglected. In this paper, we introduce a collaborative sensing platform that exploits the cameras of windshield-mounted smartphones. To demonstrate the potential of this platform, we propose several services that it can support, and prototype SignalGuru, a novel service that leverages windshield-mounted smartphones and their cameras to collaboratively detect and predict the schedule of traffic signals, enabling Green Light Optimal Speed Advisory (GLOSA) and other novel applications. Results from two deployments of SignalGuru, using iPhones in cars in Cambridge (MA, USA) and Singapore, show that traffic signal schedules can be predicted accurately. On average, SignalGuru comes within 0.66 s, for pretimed traffic signals and within 2.45 s, for traffic-adaptive traffic signals. Feeding SignalGuru's predicted traffic schedule to our GLOSA application, our vehicle fuel consumption measurements show savings of 20.3 percent, on average

Location-based Trust for Mobile User-generated Content: Applications, Challenges and Implementations

The recent explosion in shared media content and sensed data produced by mobile end-users is challenging well-established principles and assumptions in data trust models. A fundamental issue we address in this paper is how to establish some trust level in the authenticity of content created by untrusted mobile users. We advocate a secure localization and certification service that allows content producers to tag their content with with a spatial timestamp indicating its physical location. At the same time, however, our approach preserves the privacy of producers by not exposing their identity to the potential content consumers. We provide a list of existing and possible applications that would profit from such a secure localization service and sketch possible implementations of the service, highlighting advantages and drawbacks. 1

RegReS: Adaptively Maintaining a Target Density of Regional Services in Opportunistic Vehicular Networks.

Abstract—Pervasive vehicle-mounted mobile devices are increasingly common, and can be viewed as a large-scale ad hoc network on which collaborative, location-based services can be directly supported. In order to support such services within a geographic region, a certain number of computational, storage and sensing mobile devices need to be carriers of the services. This paper introduces and evaluates Region-Resident Services (RegReS), a middleware that supports such regional services by maintaining, in a fully distributed fashion, a targeted density of service carriers. Carriers collaborate opportunistically to estimate the current service density in the region and coordinate the spawning of new service carriers when necessary. Unlike previous approaches that are static, RegReS adapts to dynamic conditions such as node speed, effectively maintaining the targeted density of service carriers in highly volatile vehicular networks. Results from the ORBIT testbed, using synthetic and real bus mobility traces, show that RegReS adapts to different system configurations, preserving the desired service density with less than 16 % mean absolute error. We deployed an outdoor collaborative parking availability service atop RegReS and demonstrated RegReS’s ability to maintain the target service density with only 10 % error. I